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Integrated Learning Systems and Standardized Test Improvement 

Abstract 

This position paper considers practical concepts and ideas related to standardized test 
score improvement in general, and then considers test improvement issues relating to World 
Institute for Computer- Assisted Teaching(WICAT) Integrated Learning System (ILS) use. 
Teacher use of an ILS is discussed first, and then test "coverage" is considered. The author 
suggests that teachers are the most important variable in ELS utilization. 



Test Improvement . . . 2 



Integrated Learning Systems and Standardized Test Improvement 

Introduction 

This position paper considers some practical concepts and ideas related to standardized 
test score improvement in general, and then considers test improvement issues relating to 
World Institute for Computer-Assisted Teaching(WlCAT) Integrated Learning System (ILS) 
labs. Several years ago, WICAT Systems conducted qualitative research (see Schnitz & 
Azbell, 1991) in ILS labs across the U.S. to determine what conditions promote maximum 
learning gains. This investigation revealed that teachers are the most important variable in 
ILS utilization. Observation suggested several increasing levels of teacher "use" of ILS labs. 
These include non-participatory, novice, practitioner, integrator, and extender. 

A second important issue in test improvement involves coverage of material. Part of 
coverage includes the potential "match" between the objectives/content of the selected 
standardized test, and what is actually covered during the instructional year. Of course, the 
district syllabi may not match the national test selected. In such a case, lower test scores are 
an inevitable result. The learning environment created by the teachers use of an ILS will be 
covered first, then test "coverage" will be discussed. 

The Stages of Teacher Use of an ILS 

Five stages of teacher use have been suggested, ranging from non-participatory to 
extender (see Figure 1). Non-participatory teachers "drop-off ' their students, either literally 
or metaphorically. When a teacher is not involved in their students' learning opportunities, a 
"wall" exists that prevents the teacher from capitalizing on these opportunities. More 
importantly, the non-participatory teacher sends a non-ambiguous message that this activity is 
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Test Improvement ... 3 

not importan t. Usually in such situations, students become disinterested and little learning 
occurs. Discipline problems also increase. These effects can be mitigated by other interested 
adults (i.e., the Learning Center Manager or the Principal), however, in schools where 
teachers drop-off their students at the ILS lab, test scores are no better and are often worse 
than before the ILS was installed. 
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Figure 1 . Hypothesized level of learner performance under each instructional condition. 



Novice teachers may produce a learning environment similar to non-participatory 
teachers, not because of lack of interest in their students' learning, but because of lack of 
knowledge. The transition from novic "> higher stages depends impart on teacher desire, 
but opportunity to use the system is critical. As teachers come to the ILS lab with their 
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Test Improvement ... 4 

students, they will begin *o learn the curriculum just by helping their students in the lab. 
However, it will obviously take a full school year to get to know the different curriculum 
areas in this way. To speed the process, teachers could be given time in the lab when their 
students are not there to "play" with the available instructional options, to get to know the 
curriculum content in the various subjects, and to explore progress reports and how to use 
each type. 

However, lack of knowledge about the ILS programs is a separate issue from ■ 
establishing a positive learning environment. The student-instructor interaction established in 
an ILS lab is a critical variable in learning gains for both younger children (Clariana, 1990) 
and adults (Stephenson, 1992). The ILS lab environment usually mirrors the classroom 
environment, and very often, this dynamic either positive or negative has filtered down from 
the principal's office. A dysfunctional lab usually means a dysfunctional classroom, and vice- 
a-versa. Teacher attitude is a complex issue . Training may help. Also coaching and 
mentoring by lead teachers and by administrators can help. The introduction of an ILS lab 
into a school can serve as a cornerstone or as an impetus for change, but will not in itself 
cause change. Leaders can take advantage of the ILS to change the old rules. WICAT 
research suggests that teacher empowerment may be a cornerstone to establishing and 
maintaining a positive learning environment. 

Given time, a novice teacher becomes a Practitioner teacher. The practitioner takes 
advantage of the power of "mastery learning" that is built into ILS lessons. Practitioner 
teachers particularly impact low-able learners. These students won't do well either in the 
class or the ILS lab. Practitioners use ILS progress reports to identify and help these 
particular students, through remediation or reteaching. 

However, two curricular sequences still exist, the classroom sequence and the ILS lab 
sequence, and there is much overlap between the two. Research indicates that teaching 
material once in the classroom and once in the computer lab is more effective than either 
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Test Improvement ... 5 
classroom or computer lab alone (Dalton & Hannafin, 1988). Further, it doesn't appear to 
matter which comes first, classroom or computer. Probably, using the two different 
approaches (computer and classroom) and the double exposure to the same material makes 
the material more memorable to the learner. 

On the other hand, all students don't need a double dose of everything they leam. The 
overlap between the teacher sequence and the computer sequence is inefficient. Though 
critical for some students, the upper-ability students may become bored with this constant 
duplication of effort. For many, learning becomes a "race" through the material. It doesn't 
take long before these students realize that their reward for completing an activity is to receive 
another activity (Streibel, 1986; 1988). At this point, student "empowerment" may become 
important, because these students are likely to lose motivation in the ILS lab. Disturbingly, 
this motivation loss may impact boys and girls differently (Clariana, 1991), to the detriment 
of girls. One successful approach used by some schools involves "dynamic placement". 
This ILS feature allows a student to "jump ahead" in the ILS standard sequence if that student 
scores well on several activities in a lesson. Students that are bored from trudging through 
the ILS standard sequence usually become motivated under dynamic placement. 
Additionally, they race far ahead of their actual grade level, and this has a substantial impact 
on standardized test scores. Eventually, they reach challenging material and slow down, but 
this challenge level also maintains motivation. 

Integrator teachers manipulate the ILS lesson sequence so that it more closely matches 
or integrates with their classroom instruction. The ILS lesson sequence and the classroom 
sequence become more similar. There is a motivational factor when students do the same 
material in the classroom and in the ILS lab. Also, this situation implies that the integrator 
teacher has closely examined the computer material for that section and has selected just 
exactly what is needed. This can improve efficiency, since unuseful computer activities are 
not included. Also, it is easier to incorporate ILS percent scores into daily and weekly class 
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grades, since the computer activities directly support the current classroom unit. Grades will 
provide motivation for some students. Scored computer activities could be easily used as end 
of unit exams. 

This approach also makes it easier for the teacher to reteach or remediate low-able 
students since everyone is doing the same set of activities, and classroom remediation can 
occur with a small group rather than with individuals. However, this approach will slow 
down the high-able group. In effect, they are "brought back" to do activities they already 
have passed, and so their progress into upper grade material is delayed. 

Extender teachers have fully integrated the ILS curriculum and their classroom 
curriculum. There is one curriculum, with some topics taught only in the classroom, some 
only on the computer, and some on both. This results in a substantial time savings, since 
there is little redundancy in the lab and classroom, and increased motivation. The extra time 
gained can be used to "extend" the curriculum either into such areas as higher grade materials 
or higher-level thinking types of activities. In this learning environment, the high-able 
students may or may not "cover" as much content ground compared to their progress under 
dynamic-placement in a standard sequence. However, overall, the class average will cover 
more material, and so the class as a whole will score higher. 

Unfortunately, at all teacher levels, there is an inadvertent unfairness. The high-able 
students will either go-ahead in the curriculum, or else will have more time to do generative- 
learning type activities like creative writing. At least, comparatively, the low-able students at 
will do better than with no ILS. 

As teachers reach higher levels of ILS use, they will be forced by necessity to make 
their teaching groups smaller and smaller. To increase instructional efficiency, eventually, 
each student may need an individual education program (IEP). Tl>is is a fairly common 
approach in GED programs and at the Community College Level jand ILS systems 
complement IEP programs. This level of use is often difficult to achieve in mainstream 
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classes do to the teacher time commitment, however Chapter 1 and other special programs 
using ILS labs often achieve individual programs. 

Test Coverage 

Standardized tests are typically constructed of items covering objectives miditionally 
taught at the grade level intended to take that test. Some topics from lower and upper grades 
are included to improve discrimination. For test success, several factors must occur. (1) The 
teacher's syllabus should match the test, (2) the teacher can complete all of the syllabus 
during the year, and (3) the students remember what they have covered. 

A district syllabi for a content area or an individual teacher's curriculum plan may or 
may not match the material in the standardized test. If there is latitude in test selection, then a 
different test may be selected that more closely matches the district's syllabi, thus meeting 
local needs. In addition though, teachers should be trained to analyze tests. At a workshop, 
teachers from the same grade level could complete an examination grid for the standardized 
test used in the district. 

An examination grid consists of rows for listing the content objective of each item, and 
columns for assigning a ranking of some type (see Figure 2). The ranking system is used as 
an indication of how "deeply" the objective is tested. Usually, Bloom's taxonomy is used 
for ranking the test items. A synthesis level test item would be "deeper" than a fact level 
item, and thus students would require a deeper level of instruction to answer a synthesis 
question. Different content areas could use different ranking systems. For example, a 
science test may use the nine processes of science as ranking categories. Teachers may mark 
more than one raniring for a test. The important thing is not what level the question is, but 
rather, what must the teacher do to teach to that level. Test analysis of this type will help the 
teacher to understand "how wide and how deep" they much teach, if test improvement is 
important in the district. 
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Figure 2. Sample of an Examination Grid. 



Amount of time is always a limiting factor. The key, then, for test improvement is 
efficient covera ge of material, especially into the next grade level. For test improvement to 
occur, some topics that a teacher always covers may not be taught. As mentioned before, the 
test will include material from higher grade levels. This means that the teacher must break 
out of a narrow mind set about what is or should be taught at each grade level. For example, 
teaching 2nd graders the multiplication table (a rote memory task) if it is appropriate for those 
students will improve their standardized test scores. 

In the ILS lab, "practitioner teacher" use of dynamic placement will speed students 
along. The standard sequence of WICAT ILS w*s developed based upon current 
standardized tesfr and common textbooks. Correlations conducted by WICAT have shown 
an average of over 95% coverage of objectives on standardized tests. This indicates that the 
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WICAT standard sequence will match most tests and textbooks quite well, though of course 
it will also cover additional content not covered by some tests. For "LMS M WICAT schools, 
dynamic placement should be used. 

For WICAT "AIMS" schools, lab and classroom delivery should be more completely 
merged to avoid redundancy when it is not required (see extender discussion above). For 
difficult topics, students should receive instruction in both the classroom and the lab. Also, 
progress reports can be used to assure mastery of the content. 

ILS Prescription Test s 

An efficient approach developed by WICAT is the combination of a "prescription test" 
with objectives parallel to a school district's national, state, or local standardized test. 
Computer activities are automatically prescribed based upon the learner's test peif ormance. 
This amounts to an individual prescription for each student based upon their weaknesses. 
One implication of this approach is that students will always receive CBI activities that are 
difficult for the obvious reason that they will receive only those activities correlated to failed 
objectives on the test. 

This means that the teacher will Jje very busy in the ILS lab, compared to normal ILS 
lab use. If the teacher is not present, the WCAT test with prescription approach will 
probably be exceptionally frustrating to the students. The instructional effectiveness of this 
approach has not yet been determined, though it appears to be quite promising, given the 
proper learning environment. 

Summaw 

To maximize learning with an ILS, very complex questions about the type of learning 
environment must be answered. It would be naive to assume that an extender type 
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environment is 'test tor all conditions. In an ideal world, it may be possible to create 
individual programs for every student, and then be able to implement every program. But 
schools are faced with constraints involving time, teacher training and education, teacher and 
student abilities, class size, and student mix to name a few. 

The issue of local needs versus standardized test objectives, or even of a national 
examination, is beyond the scope of this essay. However, some simple procedures can be 
used to assure that material delivered during the year matches the school district's 
standardized test. The most obvious solution is to have a workshop to examine the test. 
Teachers involved in such a workshop will automatically adjust their teaching in-line with the 
test. This feedback or corrective function will be more important as tests evolve in the next 
decade. Computers may be used to improve test taking, allowing more than multiple-choice 
methods. 

Also, students should be trained on test taking strategies. Certain categories of 
students, particularly at-risk minority students, benefit substantially from "how to take tests" 
courses. Some students should be allowed to practice taking tests, and some students should 
be given untimed tests. Currently, some college entrance tests are being delivered by 
microcomputer. There are obvious logistic reasons for giving timed tests, however, these 
will go away as more and more, standardized tests are delivered by microcomputer. At the 
same time, the cost of test delivery may go down, since test booklets become unnecessary. 
ILS systems make the ideal delivery system, since test reports can be immediately printed 
with prescriptions. Obviously, students could also be assigned to a diagnostic/prescriptive 
assignment like the Wicat prescriptions discussed above. This could substantially increase 
instructional effectiveness. 

In summary, students will do better on tests if they know more, and if what they know 
is what is on the tes*. ILSs provide a powerful tool for teachers. 
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